Add Windows thread affinity support - #338
Conversation
|
Check-perf-impact results: (ae6918621b46271c2f10d6eb978fe95d) ❓ No new benchmark data submitted. ❓ |
PeterTh
left a comment
There was a problem hiding this comment.
Instead of duplicating the win.cc / unix.cc, I think the win version could just be reduced to a include of the helper header plus a include of the unix.cc, with an explanation (of the windows helper introducing compat with the unix impl).
| #ifdef _WIN32 | ||
| #define SKIP_UNSUPPORTED() SKIP("Affinity is not supported on Windows"); | ||
| #define SKIP_UNSUPPORTED() // SKIP("Affinity is not supported on Windows"); | ||
| #else | ||
| #define SKIP_UNSUPPORTED() | ||
| #endif |
There was a problem hiding this comment.
This whole code block (and the invocation sites) can just be removed.
| @@ -0,0 +1,105 @@ | |||
| #pragma once | |||
|
|
|||
There was a problem hiding this comment.
This file should have a comment at the start explaining its purpose (i.e. implementing unix-like affinity using windows APIs).
maybe the name should also be something like affinity_win32_adapter?
1aacd2d to
90901f4
Compare
|
|
||
| #include "log.h" | ||
|
|
||
| using pthread_t = DWORD; |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
unknown type name DWORD
|
|
||
| using pthread_t = DWORD; | ||
|
|
||
| struct cpu_set_t { |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
definition of type cpu_set_t conflicts with typedef of the same name
| uint64_t bits[WORDS] = {}; | ||
| }; | ||
|
|
||
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
|
|
||
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; | ||
|
|
||
| void CPU_ZERO(cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
|
|
||
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; | ||
|
|
||
| void CPU_ZERO(cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
while loop outside of a function
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; | ||
|
|
||
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected )
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; | ||
|
|
||
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
|
|
||
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); | ||
| void CPU_CLR(unsigned cpu, cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected )
|
|
||
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); | ||
| void CPU_CLR(unsigned cpu, cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); | ||
| void CPU_CLR(unsigned cpu, cpu_set_t* set); | ||
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected )
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); | ||
| void CPU_CLR(unsigned cpu, cpu_set_t* set); | ||
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
| void CPU_CLR(unsigned cpu, cpu_set_t* set); | ||
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); | ||
|
|
||
| int CPU_COUNT(const cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
redefinition of __sched_cpucount as different kind of symbol
| void CPU_CLR(unsigned cpu, cpu_set_t* set); | ||
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); | ||
|
|
||
| int CPU_COUNT(const cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected expression
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); | ||
|
|
||
| int CPU_COUNT(const cpu_set_t* set); | ||
| int CPU_EQUAL(const cpu_set_t* a, const cpu_set_t* b); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected )
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); | ||
|
|
||
| int CPU_COUNT(const cpu_set_t* set); | ||
| int CPU_EQUAL(const cpu_set_t* a, const cpu_set_t* b); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected parameter declarator
| static constexpr unsigned PROCS_PER_GROUP = 64; | ||
|
|
||
| struct cpu_topology_entry { | ||
| WORD group; |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
unknown type name WORD
|
|
||
| struct cpu_topology_entry { | ||
| WORD group; | ||
| WORD count; |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
unknown type name WORD
| using cpu_topology = std::vector<cpu_topology_entry>; | ||
|
|
||
| struct windows_topology_policy { | ||
| static WORD get_group_count() { return GetActiveProcessorGroupCount(); } |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
unknown type name WORD
|
|
||
| #include "log.h" | ||
|
|
||
| using pthread_t = DWORD; |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
unknown type name DWORD
|
|
||
| using pthread_t = DWORD; | ||
|
|
||
| struct cpu_set_t { |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
definition of type cpu_set_t conflicts with typedef of the same name
| uint64_t bits[WORDS] = {}; | ||
| }; | ||
|
|
||
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
|
|
||
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; | ||
|
|
||
| void CPU_ZERO(cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
|
|
||
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; | ||
|
|
||
| void CPU_ZERO(cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
while loop outside of a function
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; | ||
|
|
||
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected )
| inline constexpr unsigned CPU_SETSIZE = cpu_set_t::CPU_SETSIZE; | ||
|
|
||
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
|
|
||
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); | ||
| void CPU_CLR(unsigned cpu, cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected )
|
|
||
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); | ||
| void CPU_CLR(unsigned cpu, cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); | ||
| void CPU_CLR(unsigned cpu, cpu_set_t* set); | ||
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected )
| void CPU_ZERO(cpu_set_t* set); | ||
| void CPU_SET(unsigned cpu, cpu_set_t* set); | ||
| void CPU_CLR(unsigned cpu, cpu_set_t* set); | ||
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected unqualified-id
| void CPU_CLR(unsigned cpu, cpu_set_t* set); | ||
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); | ||
|
|
||
| int CPU_COUNT(const cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
redefinition of __sched_cpucount as different kind of symbol
| void CPU_CLR(unsigned cpu, cpu_set_t* set); | ||
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); | ||
|
|
||
| int CPU_COUNT(const cpu_set_t* set); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected expression
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); | ||
|
|
||
| int CPU_COUNT(const cpu_set_t* set); | ||
| int CPU_EQUAL(const cpu_set_t* a, const cpu_set_t* b); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected )
| int CPU_ISSET(unsigned cpu, const cpu_set_t* set); | ||
|
|
||
| int CPU_COUNT(const cpu_set_t* set); | ||
| int CPU_EQUAL(const cpu_set_t* a, const cpu_set_t* b); |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
expected parameter declarator
| static constexpr unsigned PROCS_PER_GROUP = 64; | ||
|
|
||
| struct cpu_topology_entry { | ||
| WORD group; |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
unknown type name WORD
|
|
||
| struct cpu_topology_entry { | ||
| WORD group; | ||
| WORD count; |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
unknown type name WORD
| using cpu_topology = std::vector<cpu_topology_entry>; | ||
|
|
||
| struct windows_topology_policy { | ||
| static WORD get_group_count() { return GetActiveProcessorGroupCount(); } |
There was a problem hiding this comment.
❌ clang-diagnostic-error ❌
unknown type name WORD
| // compiles unchanged. | ||
| #include "platform_specific/affinity_win32_adapter.h" | ||
|
|
||
| #include "affinity.unix.cc" |
There was a problem hiding this comment.
suspicious #include of file with .cc extension
| int CPU_COUNT(const cpu_set_t* set); | ||
| int CPU_EQUAL(const cpu_set_t* a, const cpu_set_t* b); | ||
|
|
||
| int sched_getaffinity(int pid, size_t cpusetsize, cpu_set_t* mask); |
There was a problem hiding this comment.
redundant sched_getaffinity declaration
| int sched_getaffinity(int pid, size_t cpusetsize, cpu_set_t* mask); |
| int CPU_EQUAL(const cpu_set_t* a, const cpu_set_t* b); | ||
|
|
||
| int sched_getaffinity(int pid, size_t cpusetsize, cpu_set_t* mask); | ||
| int sched_setaffinity(int pid, size_t cpusetsize, const cpu_set_t* mask); |
There was a problem hiding this comment.
redundant sched_setaffinity declaration
| int sched_setaffinity(int pid, size_t cpusetsize, const cpu_set_t* mask); |
| int sched_getaffinity(int pid, size_t cpusetsize, cpu_set_t* mask); | ||
| int sched_setaffinity(int pid, size_t cpusetsize, const cpu_set_t* mask); | ||
|
|
||
| pthread_t pthread_self(); |
There was a problem hiding this comment.
redundant pthread_self declaration
| pthread_t pthread_self(); |
|
|
||
| pthread_t pthread_self(); | ||
|
|
||
| int pthread_setaffinity_np(pthread_t thread, size_t cpusetsize, const cpu_set_t* mask); |
There was a problem hiding this comment.
redundant pthread_setaffinity_np declaration
| int pthread_setaffinity_np(pthread_t thread, size_t cpusetsize, const cpu_set_t* mask); |
| pthread_t pthread_self(); | ||
|
|
||
| int pthread_setaffinity_np(pthread_t thread, size_t cpusetsize, const cpu_set_t* mask); | ||
| int pthread_getaffinity_np(pthread_t thread, size_t cpusetsize, cpu_set_t* mask); |
There was a problem hiding this comment.
redundant pthread_getaffinity_np declaration
| int pthread_getaffinity_np(pthread_t thread, size_t cpusetsize, cpu_set_t* mask); |
| WORD group; | ||
| WORD count; |
There was a problem hiding this comment.
constructor does not initialize these fields: group, count
| WORD group; | |
| WORD count; | |
| WORD group{}; | |
| WORD count{}; |
Coverage Report for CI Build 31012551030Coverage remained the same at 95.067%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats💛 - Coveralls |
Thread pinning was previously a no-op on Windows:
thread_pinneronly emitted a warning ("Thread pinning is currently not supported on Windows.") and did not apply any affinity settings, regardless of configuration.This PR adds a real implementation by introducing a small pthread/
cpu_set_tcompatibility layer (platform_specific/affinity_win32.h/.cc). The layer maps the POSIX affinity APIs already used by Celerity (sched_getaffinity/sched_setaffinity,pthread_self,pthread_get/setaffinity_np,CPU_SET/CPU_ISSET/CPU_COUNT, etc.) to their Windows equivalents (GetActiveProcessorGroupCount,GROUP_AFFINITY,SetThreadGroupAffinity).With this compatibility layer in place,
affinity_win.ccno longer requires separate pinning logic and now closely mirrors the Linux implementation. It initializes and tears down the pinning plan in the same way and pins threads to sequential cores using the same approach.One limitation to note: Windows organizes logical processors into groups of up to 64 processors, and a single
GROUP_AFFINITYmask cannot span multiple groups. If a requested core set crosses a group boundary, the layer currently emits a warning and skips pinning instead of attempting cross-group affinity. This should not affect the common case of pinning a small number of sequential cores, but remains a limitation on systems with more than 64 logical processors.Added corresponding tests in
affinity_tests.cc.